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1 MULTIPLE WATERMARKING TECHNIQUES 

2 Field of the Invention: 

3 The present Invention relates to steganography and more particularly to the use of 

4 watenmarks to determine the authenticity and history of a particular document or 

5 image. 
6 

7 Background of the Invention: 

8 Steganographic and digital watenmarking technologies are well know. For example 

9 see U.S. Patent 5.636.292 and the extensive references cited therein. Also see 

10 copending patent applications serial number 08/327.426 which was filed 10/21/94 

1 1 and copending application serial number 08/436, 1 34 which was filed 5/8/95. 
12 

13 The technology for inserting digital watermarks in images and the technology for 

14 reading or detecting digital watermarks in images is well developed, well known and 

15 descrik>ed in detail in public literature. Furthermore, there are commercially 

16 available products which include programs or mechanisms for inserting digital 

n watemnarks into images. For example the commercially available and widely used 

18 products "Adobe Photoshop" which is marketed by Adobe Corporation of San Jose 

19 California and ''Corel Draw^ program which is mariced by Corel Corporation of 

20 Ontario Canada, include a facility for inserting digital watermarics into images. 

21 

22 The technology for making high quality copies of documents is widely available. 
.3 The technical quality of scanners and color printers has been increasing rapidly. 
24 Today for a relatively low cost one can purchase a high quality scanner and a high 
15 quality color printer. Thus, it is becoming increasingly easy to duplicate documents. 

26 The ability to create high quality copies has created a need for technology which 

27 can differentiate between original documents and copies of the original. 
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1 

2 It is known that watennarks can be used to help differentiate genuine documents 

3 from copies. However, the prior art techniques for using digital watermarks to 

4 differentiate genuine documents from copies have serious limitations. The present 

5 Invention is directed to an improved technique for using steganography and digital 

6 watemnari^ technology to facilitate differentiating original documents from copies of 

7 the original. 
8 

9 The present invention can also be used for various other purposes such as to 

10 embed multiple types of information in a single document or to provide watemnari^s 

11 which enable documents to perform special functions. . 

12 

13 Summary of the Invention: 

14 With the present invention multiple digital watermarics. each of which has different 

15 characteristics are emk>edded in a document The characteristics of the two 

16 watermarks are chosen so that each of the watermari^s will be affected in a different 

17 manner by what may subsequently happen to the document. 
18 

19 The detection process or mechanism reads the two digital watermart^s and 

10 compares their characteristics. While wear and handling may change the 

21 characteristics of the individual v^termarics, the relationship between the 

22 Characteristic of the two watermarks will never-the-less give an indication as to 

23 whether a document is an original or a copy of an original. 

24 

25 For example according to the present invention two digital watermarics in a 

26 document may have differeni energy levels. The absolute energy level of a digital 

27 watermari^ in an original image may be decreased if a document is subject to wear. 
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1 Ukewse the energy level of the digital watermark in an image may be decreased if 

2 an image is scanned and reprinted on a color printer. However, the relationship 

3 between the energy level of the two digital watemnari^s will be different in an image 

4 that has been subject to wear and in a reproduced image. Likewise if two digital 

5 watermarics are introduced into an image where the bit pattern used to construct the 

6 digital watermart^s have different patterns, the ratio between the signal to noise ratio 

7 of the watemiarics will be different in an original subject to wear and in a copy 

8 generated by scanning the original and printing the scanned image. Other 

9 characteristics of multiple digital watermarics can also be used to differentiate 

10 original documents from copies. 
11 

n Brief Description of the Figures: 

13 Figure 1 shows the paths that a document and a copy may follow. 

14 Figures 2A and 2B show a fine grain and a course grain watermaric. 

15 Figure 3A and 38 show a geometrically linear and a geometrically random 

16 assignment of pixels to a bit in a digital watermaric. 
J 7 Figure 4 illustrates a fourth emtxxliment of the invention. 
J8 

19 Detailed Description of preferred embodiments: 

20 The problem of differentiating an original document from a copy is made more 

21 difficult in situations where tlie original document is subject to being handled, worn. 

22 folded and othen^se damaged. Many original documents such as identification 

23 documents and currency are extensively handled. The wear to which such 

24 documents are subjected reduces the quality of images on the document and 

25 therefore reduces the quality of any information embedded in the document using 

26 conventional steganographic techniques. 

27 
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1 With the present invention a number of different watermarks are embedded in a 

2 document Each of the watemiarks embedded in the document, has different 

3 characteristics. All watermarks are somewhat affected when a document is 

4 subjected to wear, and all watemnarks are somewhat affected when a document is 

5 duplicated by being scanned and reprinted. However, the magnitude of the effect 

6 caused by being scanned and reprinted on watenmarics with certain characteristics 

7 is much greater than the effect on watermarics v^h different characteristics. 

8 Likewise, wear and handling of a document affects watermartcs with certain 

9 characteristics much more than it affects watermarics with different characteristics. 

10 

1 1 Thus, if multiple watermarks v^th different characteristics are inserted into a 

12 document, it is possible to differentiate a copy from an original document that has 

13 been subjected to wear by examining the ratios of characteristics of the watermarks 

14 in the image being examined. 

15 

16 In order to print a document on a color printer, the document is put through a 

17 transformation from a color space such as the RGB color space to a different color 
28 space such as the CMYK (cyan, magenta, yellow, black) color space. Such 

19 transfonmations are well know. For example see chapter 3 entitled "Color Spaces" 

20 in a book entitled "Video Demystified, A handbook for the Digital Engineer". Second 
n Edition, by Keith Jack, published by Harris Semiconductor and Hightext Publications 
22 of San Diego, Califomia. 

23 

24 When an image is transformed from one color space to another color space, noise 

25 is introduced into the image Among the reasons for this is the fact that each color 

26 space has its own distinctive gamut (or range) of colors. Where the gamut of two 

27 color spaces overiap. the conversion from one color space to another color space 
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1 can in theory be precise. However, there will be some areas which are in the gamut 

2 of one color space not in the gamut of another color space. Such situations 

3 definitely introduce noise into the conversion process. Even in areas that are in the 

4 gamut of two color spaces, conversion from one color space to another color space 

5 introduces noise because of such things as round off errors. The present invention 

6 takes advantage of the fact that if an original is copied and then a copy is printed, 

7 the image on the printed copy will have gone through several conversions to which 

8 the original will not have been subjected. For example, the conversions to which a 

9 copy may be subjected 
?o are: 

i 1 1) a document to RGB conversion (i.e. scanning the document into the computer), 

12 2) a RGB to CMYK conversion. 

J 3 3) a CMYK to copy conversion (i.e. printing the document). 

14 Any characteristics of the two digital watermarks that will be affected differently by 

15 the additional conversion process to which copies are subjected can be used to 

16 differentiate copies from an original. Since the two watermarks with different 

17 characteristics are affected in a different manner by the additional conversion step. 

18 a comparison of the characteristics of the two watemnarks in a document being 
1.9 examined will indicate if the document is an original (which has not gone through 

20 the additional conversions) or a copy v^ich has gone through the additional 

21 conversions. While the characteristics of each watermari^ will have been changed 
72 by wear and by the copying process, the comparison between the characteristics of 
23 the two watenmarks will still be able to differential a copy from an original. 

24 

25 Four embodiments of the invention are described t>elow. Each of the embodiments 

26 utilizes two v^tennarks in a document. The differences between the two 

27 watermarks in the document are as follows: 
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1 In the first embodiment: 

2 First watermark: Has fine grain 

3 Second watermark: Has a course grain 

4 In the second embodiment: 

5 First watermai k: Has geometrically linear assignment of pixels 

6 Second watemnark: Has geometrically random assignment of pixels. 

7 In the third embodiment: 

8 First watermark: Has low power 

9 Second watermark: Has higher power 

10 In the fourth embodiment: 

11 Fist watenmaric: uses standard RGB to HSI and HSI to RGB transformations 

12 Second watemnark is biased before being transfonmed from HSI to RGB. 



13 

14 Figure 1 shows the steps to which documents and copies are typically subjected. In 

!5 the normal course, a document 10 may be subjected to handling and wear 1 1 

16 resulting in a worn document 10A. Document 10 may also be scanned as illustrated 

17 by box 12. The scanning produces a digital image which can be printed as 

18 illustrated by box 13. The printed image may be subjected to handling and wear 14 

19 resulting in a copy 10B. It is noted that the document 10 may also be subject to 

20 handling and wear prior to the scanning operation 12. The task to which this 

21 invention is directed is the task of differentiating the worn document 10A from the 

22 copy 10B. 
23 

24 The document 10 includes an image (not explicitly shown) which has two digital 

25 watemnarics inserted therein. In the first emt)odiment of the invention, the first 

26 watemnark has a fine grain and the second watermartc has a course grain. The 

27 grain of the two watermarks is illustrated in Figure 2. Figure 2A shows the grain of 
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1 the first watermark and figure 2B shows the grain of the second watermark. The 

2 first watenmark uses blocks of 9 pixels (a 3 by 3 block). Each of the pixels in each 9 _ 

3 pixel block has its gray value changed by the same amount. For example Figure 2A 

4 shows that the first 9 pixel block has its gray value increase and the second 9 pixel 

5 block has its gray value decreased. The amount of increase and the selection of 

6 blocks that is increased and decreased is conventional. 
7 

8 As shown in Figure 2B. the grain of the second watermark is in blocks that are 6 

9 pixels by 6 pixels or 36 pixels. All of the pixels in each 36 pixel block are changed 

10 by the same amount. 
11 

12 In the original document 10. the two watermarks have a power ratios of 1 to 1. After 

13 wear and handling, the power of the first watenmark will be degraded somewhat 

14 more than the power of the second watenmark. For example, as illustrated in Figure 

15 1, after document 10 is subjected to handling and wear, a detector which reads the 

16 watermarks might find that the power ratio of the water marks is 1 to 2. 
17 

iS If the document 10 is scanned and the resulting digital image is printed to make a 

19 copy of the document 10, the ratio of the power of the watermarks will be affected 

20 much more than the effect of handling and yaear. For example as illustrated in 

:i Figure 1 . the power ratio of the watermarks may be 1 to 10. thereby allowing one to 

22 differentiate the worn original document 10A from the copy 10B. 

23 

24 It is noted that the mechanism for inserting watenmarks into an image is well known 

25 as is the technique for reading a v\«tenmark and using con^elation techniques to 

26 determine the signal to noise ratio (i.e. the power) of a watemiark. 
27 
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1 Figures 3A and 3B shown an alternative technique for implementing the present 

2 invention. In the second eml)odiment of the invention, the two watermarks inserted 

3 into the image on a document have different patterns of assigning pixels to the bits 

4 of the number v^^hich the vratermark represents. The first watenmark utilizes a 

5 geometrically linear assignment of pixels to each bit. For example Figure 3A shows 

6 an image that has 500 by 500 pixels. Considering a watermark with 50 bits, each 

7 bit of the watermark would have 5000 pixels assigned to represent that bit. A linear 

8 assignment could have each fifth bit in each row (100 bits per row) and each fifth 

9 row (50 rows) assigned to each bit of the watermark. Thus 5000 pixels would be 

10 assigned to each bit in a very orderly or linear manner, 
n 

12 In the second watermark the pixels would be assigned to each bit in a random 

13 manner as shown in Figure SB. Each bit in the watemiark would still have 5000 

14 assigned bits; however, the pixels would be a random location over the image. 

15 Naturally it should be understood that Figure 3A and 38 illustrate how pixels are 

16 assigned to one bit of the watermark. The other bits of the watenrtiarks would have 

17 pixels assigned in a similar manner. 
18 

iSf Similar to the first embodiment of the invention, the watemiark v^th a linear 

20 assignment of pixels and the watenmark with a random assignment of pixels would 

21 be affected differently by handling and wear on the original document than they 

22 would be by being scanned and reprinted. 

23 

14 The third embodiment of the invention described herein utilizes watermarics which 

25 have different power levels. Handling and wear as contrasted to scanning and 

26 printing would affect a watermaric witti a low power level differentiy than a water 

27 mari^ with a high power level. Watermarics with different pov^r levels can be 
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1 inserted into a document in order to practice tlie present invention utilizing 

2 commercially available programs such as Adobe Photoshop or Corel Draw. In the 

3 Adobe Photoshop and Corel Draw programs, the power or intensity of the 

4 watermark can be adjusted by setting a simple control setting in the program. 
5 

6 The fourth embodiment of the invention introduces different characteristics into two 

7 watermarks by modifications made to one of the watermarks during the initial step 

8 during v^ich the watermarks: are introduced into an image. The operation of the 

9 fourth embodiment r^n be explained as shown in Figure 4. First as illustrated by 
JO equation 1 there is a conversion from RGB to HSI as is conventional. This is 

1 1 illustrated by equation 1 . As illustrated by equation 2, the first watermark is inserted 

12 into the image in a conventional manner by modifying the I value in the HSI 

13 representation of the image using the first watermark values (designated as WM1 
U A). A first RGB value designated RGB(1) is then calculated using a conventional 
15 transformation designated T. As indicated by equation 3, the second watermark 
J6 WM2 is then biased toward a particular color and the biased watermark is then 

17 combined with the HSI values and transformed to a second set of RGB values 

18 designated RGB(2). Finally as indicated by equation 4, the values RGB(1) and 

19 RGB(2) are combined to fomi the watermarked image designated RGB(F). 

20 

2l~ The transform used to go from RGB to HSI color space (indicated in equation 1 in 

22 Figure 4) can be anyone of a variety of known other techniques. For example, the 

23 RGB to HSI conversion can be one of the techniques explained in the above 

24 referenced text book such as the following: (which assumes that RGB and Intensity 

25 have a value range of 0 to I and that Red equals O"*): 

26 First calculate: 

27 M = max (R.G.B) 
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1 m = min (R.G.B) 

2 r= {M-R)/(M-m) 

3 g = (M-G) /M-n) 

4 b = (M-B)/(M-m) 

5 Then calculate I, S, and H as follows: 

6 a) \ = (M + M)/2 

7 b) lfM = mthenS = 0andH = 180 

8 if l< or = 0.5 then S = (M-m)/(M+m) 

9 if I > 0.5 then S = (M-m) / (2-M-m) 
!0 c) if R = M then H = 60 (b-g) 

11 ifG = M then H = 60(2 +r-b) 

12 if B = M then H = 60(4 + g - r) 

13 if H>or=360thenH = h- 360 

14 if H<0thenH = H>360 

15 The first watermark in inserted into the RGB values in a conventional manner by 

16 modifying the I value of appropriate pixels so as to combine the watemnark A values 

17 with HSI values. This is indicated by equation 2 in Figure 4. Next as indicated by 

18 equation 3 in Figure 4, the HSI values are converted to RGB values using a 

19 transform T". The transform T" can be conventional and it can for example be 

20 done as follows: 
?l First calculate: 

2 if l<or = 0.5thenM = l(l + S) 

23 ifl>0.5thenM=l + S-IS 

34 m = 21 - M 

-!5 if S = 0 then R = G = B = landH = 180" 

36 Then calculate R. G and B as follows: 
a) if H < 60 then R = M 
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1 if H < 120 then R = m + ((f^m) / ((120- H) / 60)) 

2 if H < 240 then R = m 

3 if H < 300 then R = m + ((M - m) / ((H - 240 / 60)) 

4 otherwise R = M 

5 

6 b) if H < 60 then G = m + ((M-m)/ ( H/60)) 

7 if H<180thenG = M 

8 if H '< 240 then G = m + ((M - m) / ((240 - H_ / 60)) 

9 otherwise G = m 

10 c) if H < 120 then B = m 

11 if H < 180 then B = m + ((M -m) / ((H-120/60)) 

12 if H < 300 then B = M 

13 otherwise B = m ((M-m)/ ((360 -H)/ 60)) 

14 

<3 Next the values which represent a second watermark are used to calculate a 

16 second set of RGB values designated RGB2. In order to calculate RGB2, the 

17 values of H and S are modified so that they are slightly biased toward a particular 

18 color designated H1 and SI New values for H and S are calculated as follows: 

19 (Note, HI must be beivt^n 0 and 360, S1 must be non-negative and can be 

20 between 0 and 1 and X is a value between 0 and 1) 

21 Calculate new values for H and S as follows: 

22 If H > HI then H = H - (H - H1) x 

23 else H = H + (HI -H)x 

24 lfS>S1then S=S-(S-S1)x 

25 else S = S+(S1-S)x 
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1 Next add the second watermark to the values of HSI and transfonn these values to 

2 the RGB color space as inaicated by equation 3 The transformation from HSI color 

3 space to RGB color space is done as previously indicated. 

4 

5 Finally as indicated by equation 4 in Figure 4. the final RGB value (designated 

6 RGBF) is calculated by combining the values of RGB1 and RGB2. This 

7 combination can be done in a variety of known ways. 
8 

) It is noted that in the .^oove example the difference between the transformation 

»0 used for the first and the secx>nd watennarks involves biasing the values of H and S. 

• I Alternatively a wide variety of different changes could also be made. The key to 

12 this fourth embodiment of the invention is that in effect a different transformation is 

13 used for the first and the second watermarks 
u 

15 While four embodiments of the invention have l>een shown herein , it should be 

16 understood that many other characteristics and attributes of a digital watemnark 

(7 could be used to practice thB present invention in addition to the characteristics and 

!f; attributes described herein. Furthermore other known digital watermarking 

:9 techniques can be used toaether with and applied to the digital watermarks used for 

T) the present invention. It is also noted that v^ile in the above examples only two 

21 watermarks v^re used, in some situations one could use three, four five or more 

12 watermarks. That is. the enbodiments of the invention specifically described herein 

23 Utilize two watermarks. It should be understood that any number of watermarks 

24 could be utilized in l«i:e manner. Furthermore while the embodiments shown herein 

25 utilize two separate Wr:term3:ks. the two watermarks used to practice the present 

26 invention could be combined into one watermark which has a plurality of separate 

27 identifiable and measurable characteristics. 
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2 It is noted that while the present invention utilizes multiple watermarks with different 

3 characteristics to differentiate original documents from copies of the original, one 

4 can also utilizes multiple watermarks with different characteristics for other reasons. 

5 Documents may include multiple similar watermarks in addition to the watermarks 

6 which have different characteristics according to the present invention. As used 

7 herein, in general, the term "document" refers to a physical entity. 
8 

9 While the present invention has been described with respect to four specific 

10 embodiments of the invention, it should be understood that various changes in 

1 1 fonma and detail could be made without departing from the spirit and scope of the 

12 invention. The scope of the present invention is limited only by the appended 

13 claims. 
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1 I Claim 

2 1) A document which has embedded therein a first digital watermark having a first 

3 set of characteristics and a second watermark having a set of characteristics which 

4 differ from said first set of characteristics. 

5 

6 2) The document recited in claim 1 wherein said first watermari^ has a different 

7 energy level from said second watermark. 
8 

9 3) The document reciisd in claim 1 v^erein said first and second watermarks have 

10 bit patterns and wherein the bit patterns which comprise said first watermart^ are 

1 1 different from the bit patterns which comprise said second watenmari^ 

i2 

13 4) A method of creating a watermari^ed image which comprises the steps of: 

14 modifying an HSI (hue. saturation, Intensity) representation of an image to imbed a 

15 first watermariced in said image and create a first watermariced image, 

16 transforming said first watemnari^ed image to the RGB (red. green, blue) color 

17 space. 

18 biasing the values v^ich represent a second watermari^ toward a particular color, 

19 modifying a said HSI values to imbed said biased second watenmari^ed in said 

20 image and create a second watermariced image. 

21 ' ti^nsforrntng said second watennnaTked image to the RGB color space, and 

22 combining the values first and second watenmartced images to create a final 

23 watermarked image. 

24 

25 5) The method redtPd in claim 4 v^erein said biasing is toward a particular color. 
26 
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1 6) A method of differentiating copies of an original document from the original 

2 document comprising, said document containing a first digital watermartc which has 

3 a first set of characteristics and a second digital watermari< which has a second set 

4 of characteristics. 

5 reading said first and second watermartcs from said original document and 

6 comparing the resultant values to generate a first set of results. 

7 reading said first and second watermarks from said copy of said original document 

8 and comparing the resulting values to generate a second set of results, and 

9 using differences between said first and second sets of results to differentiate an 

10 6qginal document from a copy of said original document. 
II 

12 7 ) The method recited in claim 6 wherein said first digital watenmartc has a first 

13 energy level and said second watermark has a second energy level. 
14 

15 8) The method recited in claim 6 wherein said first v/atennark has a first bit pattern 

J 6 . and said second watermaric has a different bit pattern. 

i7 

(8 9) The method recited in claim 6 where both said original document and said copy 

19 have been subjected to wear. 

20 

"21 ~ 10) The document redled in claim 1 wherein said second watennark was been 
22 biased toward a particular color before it was inserted in said document. 
23 

11 1 1) A method of differontiating an original document from a copy of said document. 

25 said document having first and second watermartcs imbedded therein, comprising 

26 the steps of comparing the characteristics of the said first and second watermarics in 
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1 said original documeni and comparing the characteristics of said first and second 

2 watermarks in said copy. 
3 

4 12) the method recited In claim 1 1 wherein said first and second watennarics have 

5 different grain structures. 
6 

7 13) The method recited in claim 1 1 wherein said first and second watemnarks have 

8 different intensity levels. 
9 

10 14) The method recited in claim 12 wherein said first and second watermari<s have 

1 1 different characteristics. 

12 

13 15) A document which contains an image which has embedded therein a plurality of 

14 watenmarks, each of said watermarks having characteristics which differ from each 

15 other. 
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Document with two watermarks 
One Watermark with fine grain 
One Watermark with coarse grain 
Power ratio of watermarics 1 to 1 
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Scanner ^ 
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Document that has been 
subjected to wear 
Power ratios of watermarks 1 to 2 



Printer 
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FIG. 1 



Copy of Original Document 
Power ratio of watermarks 1 to 10 
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FIG. 2A 

Watermark with a fine grain (each block of pixels is 3 by 3) 
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FIG. 2B 

Watermark with a coarse grain (each block of pixels is 6 by 6) 
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FIG. 3A Geometrically linear assignment of pixels to each bit 
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FIG. 3B Geometrically random assignment of pixels to each bit 
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(1) RGB ►HSI 



(2) First Watermark 



HSI + WMIA ►RGBl 



(3) Second Watermark 



HSI +Biased WM2A ► RGB2 



(4) Final image (RGB 1 + RGB2) / 2 = RGBF 



